Natural Turn-taking Needs No Manual: Computational Theory and Model, from Perception to Action

t-minus-460 msec Beth and Alan are sitting at a Fifth Avenue outdoors restaurant in Manhattan. Alan is telling Beth an exciting story about his vacation in Nice. Alan presents the story through gesture and speech. Then Beth’s arm starts moving and her neck stiffens. We, the viewers, know that she’s surprised to see an elephant in the middle of Manhattan, and that in 460 milliseconds her arm and hand motion will turn into a welldefined deictic gesture, her eyebrows will rise, and her mouth will open with surprise, at which point Alan will most certainly recognize the signs and look over at the elephant. But right now, at t-minus-460 milliseconds, Beth’s gesture is barely recognizable as a communicative action, so Alan doesn’t know for sure. And thus, before that all happens, in the next 460 milliseconds, Alan has to decide what to do about Beth’s behavior. Should he stop telling his story? Or should he go on, in case Beth is simply adjusting her jacket?